Test Maintenance for Machine Learning Systems: A Case Study in the Automotive Industry

Machine Learning (ML) systems have seen widespread use for automated decision making. Testing is essential to ensure the quality of these systems, especially safety-critical autonomous systems in the automotive domain. ML systems introduce new challenges with the potential to affect test maintenance, the process of updating test cases to match the evolving system. We conducted an exploratory case study in the automotive domain to identify factors that affect test maintenance for ML systems, as well as to make recommendations to improve the maintenance process. Based on interview and artifact analysis, we identified 14 factors affecting maintenance, including five especially relevant for ML systems—with the most important relating to non-determinism and large input spaces. We also proposed ten recommendations for improving test maintenance, including four targeting ML systems—in particular, emphasizing the use of test oracles tolerant to acceptable non-determinism. The study’s findings expand our knowledge of test maintenance for an emerging class of systems, benefiting the practitioners testing these systems

User Label Leakage from Gradients in Federated Learning

Federated learning enables multiple users to build a joint model by sharing their model updates (gradients), while their raw data remains local on their devices. In contrast to the common belief that this provides privacy benefits, we here add to the very recent results on privacy risks when sharing gradients. Specifically, we propose Label Leakage from Gradients (LLG), a novel attack to extract the labels of the users' training data from their shared gradients. The attack exploits the direction and magnitude of gradients to determine the presence or absence of any label. LLG is simple yet effective, capable of leaking potential sensitive information represented by labels, and scales well to arbitrary batch sizes and multiple classes. We empirically and mathematically demonstrate the validity of our attack under different settings. Moreover, empirical results show that LLG successfully extracts labels with high accuracy at the early stages of model training. We also discuss different defense mechanisms against such leakage. Our findings suggest that gradient compression is a practical technique to prevent our attack

Induction and Exhaustion of Lymphocytic Choriomeningitis Virus–specific Cytotoxic T Lymphocytes Visualized Using Soluble Tetrameric Major Histocompatibility Complex Class I–Peptide Complexes

This study describes the construction of soluble major histocompatibility complexes consisting of the mouse class I molecule, H-2Db, chemically biotinylated β2 microglobulin and a peptide epitope derived from the glycoprotein (GP; amino acids 33–41) of lymphocytic choriomeningitis virus (LCMV). Tetrameric class I complexes, which were produced by mixing the class I complexes with phycoerythrin-labeled neutravidin, permitted direct analysis of virus-specific cytotoxic T lymphocytes (CTLs) by flow cytometry. This technique was validated by (a) staining CD8+ cells in the spleens of transgenic mice that express a T cell receptor (TCR) specific for H-2Db in association with peptide GP33–41, and (b) by staining virus-specific CTLs in the cerebrospinal fluid of C57BL/6 (B6) mice that had been infected intracranially with LCMV-DOCILE. Staining of spleen cells isolated from B6 mice revealed that up to 40% of CD8+ T cells were GP33 tetramer+ during the initial phase of LCMV infection. In contrast, GP33 tetramers did not stain CD8+ T cells isolated from the spleens of B6 mice that had been infected 2 mo previously with LCMV above the background levels found in naive mice. The fate of virus-specific CTLs was analyzed during the acute phase of infection in mice challenged both intracranially and intravenously with a high or low dose of LCMV-DOCILE. The results of the study show that the outcome of infection by LCMV is determined by antigen load alone. Furthermore, the data indicate that deletion of virus-specific CTLs in the presence of excessive antigen is preceded by TCR downregulation and is dependent upon perforin

Parameters for the mathematical modelling of Clostridium difficile acquisition and transmission: a systematic review

INTRODUCTION: Mathematical modelling of Clostridium difficile infection dynamics could contribute to the optimisation of strategies for its prevention and control. The objective of this systematic review was to summarise the available literature specifically identifying the quantitative parameters required for a compartmental mathematical model of Clostridium difficile transmission. METHODS: Six electronic healthcare databases were searched and all screening, data extraction and study quality assessments were undertaken in duplicate. Results were synthesised using a narrative approach. RESULTS: Fifty-four studies met the inclusion criteria. Reproduction numbers for hospital based epidemics were described in two studies with a range from 0.55 to 7. Two studies provided consistent data on incubation periods. For 62% of cases, symptoms occurred in less than 4 weeks (3-28 days) after infection. Evidence on contact patterns was identified in four studies but with limited data reported for populating a mathematical model. Two studies, including one without clinically apparent donor-recipient pairs, provided information on serial intervals for household or ward contacts, showing transmission intervals of <1 week in ward based contacts compared to up to 2 months for household contacts. Eight studies reported recovery rates of between 75%-100% for patients who had been treated with either metronidazole or vancomycin. Forty-nine studies gave recurrence rates of between 3% and 49% but were limited by varying definitions of recurrence. No study was found which specifically reported force of infection or net reproduction numbers. CONCLUSIONS: There is currently scant literature overtly citing estimates of the parameters required to inform the quantitative modelling of Clostridium difficile transmission. Further high quality studies to investigate transmission parameters are required, including through review of published epidemiological studies where these quantitative estimates may not have been explicitly estimated, but that nonetheless contain the relevant data to allow their calculation

NIST Interlaboratory Study on Glycosylation Analysis of Monoclonal Antibodies: Comparison of Results from Diverse Analytical Methods

Glycosylation is a topic of intense current interest in the development of biopharmaceuticals because it is related to drug safety and efficacy. This work describes results of an interlaboratory study on the glycosylation of the Primary Sample (PS) of NISTmAb, a monoclonal antibody reference material. Seventy-six laboratories from industry, university, research, government, and hospital sectors in Europe, North America, Asia, and Australia submit- Avenue, Silver Spring, Maryland 20993; 22Glycoscience Research Laboratory, Genos, Borongajska cesta 83h, 10 000 Zagreb, Croatia; 23Faculty of Pharmacy and Biochemistry, University of Zagreb, A. Kovacˇ ic´ a 1, 10 000 Zagreb, Croatia; 24Department of Chemistry, Georgia State University, 100 Piedmont Avenue, Atlanta, Georgia 30303; 25glyXera GmbH, Brenneckestrasse 20 * ZENIT / 39120 Magdeburg, Germany; 26Health Products and Foods Branch, Health Canada, AL 2201E, 251 Sir Frederick Banting Driveway, Ottawa, Ontario, K1A 0K9 Canada; 27Graduate School of Advanced Sciences of Matter, Hiroshima University, 1-3-1 Kagamiyama Higashi-Hiroshima 739–8530 Japan; 28ImmunoGen, 830 Winter Street, Waltham, Massachusetts 02451; 29Department of Medical Physiology, Jagiellonian University Medical College, ul. Michalowskiego 12, 31–126 Krakow, Poland; 30Department of Pathology, Johns Hopkins University, 400 N. Broadway Street Baltimore, Maryland 21287; 31Mass Spec Core Facility, KBI Biopharma, 1101 Hamlin Road Durham, North Carolina 27704; 32Division of Mass Spectrometry, Korea Basic Science Institute, 162 YeonGuDanji-Ro, Ochang-eup, Cheongwon-gu, Cheongju Chungbuk, 363–883 Korea (South); 33Advanced Therapy Products Research Division, Korea National Institute of Food and Drug Safety, 187 Osongsaengmyeong 2-ro Osong-eup, Heungdeok-gu, Cheongju-si, Chungcheongbuk-do, 363–700, Korea (South); 34Center for Proteomics and Metabolomics, Leiden University Medical Center, P.O. Box 9600, 2300 RC Leiden, The Netherlands; 35Ludger Limited, Culham Science Centre, Abingdon, Oxfordshire, OX14 3EB, United Kingdom; 36Biomolecular Discovery and Design Research Centre and ARC Centre of Excellence for Nanoscale BioPhotonics (CNBP), Macquarie University, North Ryde, Australia; 37Proteomics, Central European Institute for Technology, Masaryk University, Kamenice 5, A26, 625 00 BRNO, Czech Republic; 38Max Planck Institute for Dynamics of Complex Technical Systems, Sandtorstrasse 1, 39106 Magdeburg, Germany; 39Department of Biomolecular Sciences, Max Planck Institute of Colloids and Interfaces, 14424 Potsdam, Germany; 40AstraZeneca, Granta Park, Cambridgeshire, CB21 6GH United Kingdom; 41Merck, 2015 Galloping Hill Rd, Kenilworth, New Jersey 07033; 42Analytical R&D, MilliporeSigma, 2909 Laclede Ave. St. Louis, Missouri 63103; 43MS Bioworks, LLC, 3950 Varsity Drive Ann Arbor, Michigan 48108; 44MSD, Molenstraat 110, 5342 CC Oss, The Netherlands; 45Exploratory Research Center on Life and Living Systems (ExCELLS), National Institutes of Natural Sciences, 5–1 Higashiyama, Myodaiji, Okazaki 444–8787 Japan; 46Graduate School of Pharmaceutical Sciences, Nagoya City University, 3–1 Tanabe-dori, Mizuhoku, Nagoya 467–8603 Japan; 47Medical & Biological Laboratories Co., Ltd, 2-22-8 Chikusa, Chikusa-ku, Nagoya 464–0858 Japan; 48National Institute for Biological Standards and Control, Blanche Lane, South Mimms, Potters Bar, Hertfordshire EN6 3QG United Kingdom; 49Division of Biological Chemistry & Biologicals, National Institute of Health Sciences, 1-18-1 Kamiyoga, Setagaya-ku, Tokyo 158–8501 Japan; 50New England Biolabs, Inc., 240 County Road, Ipswich, Massachusetts 01938; 51New York University, 100 Washington Square East New York City, New York 10003; 52Target Discovery Institute, Nuffield Department of Medicine, University of Oxford, Roosevelt Drive, Oxford, OX3 7FZ, United Kingdom; 53GlycoScience Group, The National Institute for Bioprocessing Research and Training, Fosters Avenue, Mount Merrion, Blackrock, Co. Dublin, Ireland; 54Department of Chemistry, North Carolina State University, 2620 Yarborough Drive Raleigh, North Carolina 27695; 55Pantheon, 201 College Road East Princeton, New Jersey 08540; 56Pfizer Inc., 1 Burtt Road Andover, Massachusetts 01810; 57Proteodynamics, ZI La Varenne 20–22 rue Henri et Gilberte Goudier 63200 RIOM, France; 58ProZyme, Inc., 3832 Bay Center Place Hayward, California 94545; 59Koichi Tanaka Mass Spectrometry Research Laboratory, Shimadzu Corporation, 1 Nishinokyo Kuwabara-cho Nakagyo-ku, Kyoto, 604 8511 Japan; 60Children’s GMP LLC, St. Jude Children’s Research Hospital, 262 Danny Thomas Place Memphis, Tennessee 38105; 61Sumitomo Bakelite Co., Ltd., 1–5 Muromati 1-Chome, Nishiku, Kobe, 651–2241 Japan; 62Synthon Biopharmaceuticals, Microweg 22 P.O. Box 7071, 6503 GN Nijmegen, The Netherlands; 63Takeda Pharmaceuticals International Co., 40 Landsdowne Street Cambridge, Massachusetts 02139; 64Department of Chemistry and Biochemistry, Texas Tech University, 2500 Broadway, Lubbock, Texas 79409; 65Thermo Fisher Scientific, 1214 Oakmead Parkway Sunnyvale, California 94085; 66United States Pharmacopeia India Pvt. Ltd. IKP Knowledge Park, Genome Valley, Shamirpet, Turkapally Village, Medchal District, Hyderabad 500 101 Telangana, India; 67Alberta Glycomics Centre, University of Alberta, Edmonton, Alberta T6G 2G2 Canada; 68Department of Chemistry, University of Alberta, Edmonton, Alberta T6G 2G2 Canada; 69Department of Chemistry, University of California, One Shields Ave, Davis, California 95616; 70Horva´ th Csaba Memorial Laboratory for Bioseparation Sciences, Research Center for Molecular Medicine, Doctoral School of Molecular Medicine, Faculty of Medicine, University of Debrecen, Debrecen, Egyetem ter 1, Hungary; 71Translational Glycomics Research Group, Research Institute of Biomolecular and Chemical Engineering, University of Pannonia, Veszprem, Egyetem ut 10, Hungary; 72Delaware Biotechnology Institute, University of Delaware, 15 Innovation Way Newark, Delaware 19711; 73Proteomics Core Facility, University of Gothenburg, Medicinaregatan 1G SE 41390 Gothenburg, Sweden; 74Department of Medical Biochemistry and Cell Biology, University of Gothenburg, Institute of Biomedicine, Sahlgrenska Academy, Medicinaregatan 9A, Box 440, 405 30, Gothenburg, Sweden; 75Department of Clinical Chemistry and Transfusion Medicine, Sahlgrenska Academy at the University of Gothenburg, Bruna Straket 16, 41345 Gothenburg, Sweden; 76Department of Chemistry, University of Hamburg, Martin Luther King Pl. 6 20146 Hamburg, Germany; 77Department of Chemistry, University of Manitoba, 144 Dysart Road, Winnipeg, Manitoba, Canada R3T 2N2; 78Laboratory of Mass Spectrometry of Interactions and Systems, University of Strasbourg, UMR Unistra-CNRS 7140, France; 79Natural and Medical Sciences Institute, University of Tu¨ bingen, Markwiesenstrae 55, 72770 Reutlingen, Germany; 80Bijvoet Center for Biomolecular Research and Utrecht Institute for Pharmaceutical Sciences, Utrecht University, Padualaan 8, 3584 CH Utrecht, The Netherlands; 81Division of Bioanalytical Chemistry, Amsterdam Institute for Molecules, Medicines and Systems, Vrije Universiteit Amsterdam, de Boelelaan 1085, 1081 HV Amsterdam, The Netherlands; 82Department of Chemistry, Waters Corporation, 34 Maple Street Milford, Massachusetts 01757; 83Zoetis, 333 Portage St. Kalamazoo, Michigan 49007 Author’s Choice—Final version open access under the terms of the Creative Commons CC-BY license. Received July 24, 2019, and in revised form, August 26, 2019 Published, MCP Papers in Press, October 7, 2019, DOI 10.1074/mcp.RA119.001677 ER: NISTmAb Glycosylation Interlaboratory Study 12 Molecular & Cellular Proteomics 19.1 Downloaded from https://www.mcponline.org by guest on January 20, 2020 ted a total of 103 reports on glycan distributions. The principal objective of this study was to report and compare results for the full range of analytical methods presently used in the glycosylation analysis of mAbs. Therefore, participation was unrestricted, with laboratories choosing their own measurement techniques. Protein glycosylation was determined in various ways, including at the level of intact mAb, protein fragments, glycopeptides, or released glycans, using a wide variety of methods for derivatization, separation, identification, and quantification. Consequently, the diversity of results was enormous, with the number of glycan compositions identified by each laboratory ranging from 4 to 48. In total, one hundred sixteen glycan compositions were reported, of which 57 compositions could be assigned consensus abundance values. These consensus medians provide communityderived values for NISTmAb PS. Agreement with the consensus medians did not depend on the specific method or laboratory type. The study provides a view of the current state-of-the-art for biologic glycosylation measurement and suggests a clear need for harmonization of glycosylation analysis methods. Molecular & Cellular Proteomics 19: 11–30, 2020. DOI: 10.1074/mcp.RA119.001677.L

Efficient Anonymous Group Communication

This dissertation addresses the important challenge of efficiency in anonymous communication. Solving this challenge is essential to provide anonymity in group communication. Every exchanged message leaks metadata: this information describes the communication itself with, among others, sender, recipients, frequency of the communication. While the law protects this information, it is often published and misused with consequences for the participants of the communication—often consequences particular for the senders of information. Anonymous communication systems like Tor break the link between senders and recipients of messages and diminish emerging metadata. However, their design requires duplicating messages for all recipients early, mostly at the sender itself. With that, the system has to handle an unnecessary burden of processing identical messages. This dissertation contributes a novel mechanism that establishes communication groups such that the message duplication is pushed as close to the recipients as possible. This dissertation also shows that this efficiency improvement does not come at costs of anonymity. Moreover, the group establishment mechanism increases the robustness of the communication against users that leave and join the communication system. To encounter the additional information leakage, different mechanisms to share routing information are introduced and discussed under the angle of efficiency and anonymity. To also protect senders of messages, this dissertation adapts Dining-Cryptographer networks to enable sender anonymity with an adjustable trade-off between efficiency and anonymity. The scientific contributions of this dissertation fall into the following categories: - Efficient Communication Overlays: A novel overlay establishment mechanism is presented. This mechanism adapts Ant Colony Optimization (ACO) to connect senders and recipients with a reduced number of connections while the anonymity remains stable. - Reliable Communication under Churn: Churn often disrupts communication overlays, this thesis proposes a mechanism to counter this disruptions and increase the robustness of the communication. For that, the ACO-based mechanism utilizes residual pheromones to reconnect subjects to the communication overlay. - Routing Information Exchange considering Efficiency and Anonymity: Four methods to share routing information, namely successor lists, successor lists with multi-layer encryption, Bloom filters, and distributed lookup tables, are introduced into the anonymous communication setting, discussed and evaluated with respect to their properties concerning efficiency and anonymity. - Efficient and Effective Sender Protection: A novel mechanism based on asymmetric dining-cryptographer networks (ADCnets) is proposed to improve the efficiency of sender protection without degrading anonymity over time. Moreover, the trade-off between efficiency and anonymity can be controlled. Evaluation The developed mechanisms have been extensively evaluated using a combination of simulations and formal arguments. For this evaluation, a graph-based simulation model has been developed enabling to analyze the improvement of the ACO-based communication overlays over conventional overlays. An extensive simulation identified valuable parameter combinations for the ACO mechanism, leading to communication overlays with an efficiency improvement of up to 40%. This efficiency improvement increases the communication delay only by up to 2 extra hops. The calculation of the achieved anonymity degree indicates no loss of anonymity in comparison to conventional overlays; even more, the anonymity degree does even improve. Under churn, the robustness of the communication also increases by 30% in comparison to conventional overlays. The four approaches to routing information exchange are discussed and compared using formal arguments. The evaluation enables to select the appropriate mechanism based on the requirements for memory and communication efficiency and anonymity for the desired application scenario. The novel adapted ADCnets enable sender anonymity with configurable efficiency and anonymity. As such, they are superior to current approaches implementing cover traffic. A simulation study shows that cover traffic randomization improves efficiency at the cost of anonymity. The proposed ADCnets have been evaluated using formal arguments that demonstrate the efficiency improvement and the preservation of anonymity. As both efficiency and anonymity are conflicting, they cannot be achieved at the same time; however, the proposed ADCnets enable to balance efficiency and anonymity to the requirements of the application scenario

Asymmetric DCnets for Effective and Efficient Sender Anonymity

Emerging connected devices lead to ubiquitous communication in which anonymity and efficiency gain additional importance. In this paper, we show that current measures for sender anonymity are not sufficient and propose a new approach based on DCnets. The novel ADCnet mechanism establishes local DCnet groups that communicate asymmetrically and hide senders with lower communication overhead in comparison to cover traffic-based anonymization and classical DCnets. This paper presents concepts for the initialization and the group formation of ADCnets. The novel mechanism of ADCnets is evaluated w.r.t. anonymity and efficiency. We show that ADCnets provide DCnet-like anonymity while massively improving efficiency

MODS: Modularly Operated Digital Signage

Signage is transitioning from static analogue signs towards Digital Signage (DS), which introduces a variety of benefits. Among those are remote-maintenance, supporting dynamic content like videos and animations, and the simplification of updating content. However, DS solutions, despite being ubiquitous, are often tailored to specific use-cases, which limits their re-usability and updateability in case of severe changes to their environment. For instance, digital door signs for office spaces may become unusable if the office space is reorganized as storage or seminar rooms. Coping with such changes may result in additional costs since new DS solutions need to be purchased or in-depth changes to the software of the currently deployed DS solution are required. To address these problems we propose the Modularly Operated Digital Signage (MODS) framework, facilitating the dynamic changing of DS functionality in a modular fashion. We present the frameworks modular concept and describe its individual components. Subsequently, we briefly elaborate on the properties of the currently implemented modules. Additionally, we discuss the conducted pre-study to receive a first indicator for the usability of the framework

Monotone Sampling of Networks

Determining the graph-theoretic properties of large real-world networks like social, computer, and biological networks, is a challenging task. Many of those networks are too large to be processed e ciently and some are not even available in their entirety. In order to reduce the size of available data or collect a sample of an existing network, several sampling algorithms were developed. They aim to produce samples whose properties are close to the original network. It is unclear what sample size is su cient to obtain a sample whose properties can be used to estimate those of the original network. This estimation requires sampling algorithms that produce results that converge smoothly to the original properties since estimations based on unsteady data are unreliable. Consequently, we eval- uate the monotonicity of sampled properties while increasing the sample size. We provide a ranking of common sampling algorithms based on their monotonicity of relevant network properties using the results from four nework classes.&nbsp